An a.e. divergent martingale that converges in probability

TD ( X ) Converges with Probability 1

The methods of temporal differences (Samuel, 1959; Sutton, 1984, 1988) allow an agent to learn accurate predictions of stationary stochastic future outcomes. The learning is effectively stochastic approximation based on samples extracted from the process generating the agent's future. Sutton (1988) proved that for a special case of temporal differences, the expected values of the predictions co...

In discrete time a local martingale is a martingale under an equivalent probability measure

1 Result and Discussion We consider a discrete-time infinite horizon model with an adapted d-dimensional process S = (S t) given on a stochastic basis (Ω, F, F = (F t) t=0,1,... , P). The notations used: M(P), M loc (P) and P are the sets of d-dimensional martingales, local martingales and predictable (i.e. (F t−1)-adapted) processes; H · S t = j≤t H j ∆S j. To our knowledge, this result was ne...

An Observer that Converges in Finite Time Due to Measurement-based State Updates

This paper presents a new observer that estimates the exact state of a linear continuous-time system in predetermined finite time. The finite convergence time of the proposed observer is achieved by updating the observer state based on the difference between the measured output and the estimated output at discrete time instants. Simulation results are presented to illustrate the convergence beh...

1 Martingale Representation and All That

An Exponential Martingale Equation

We prove an existence of a unique solution of an exponential martingale equation in the class of BMO martingales. The solution is used to characterize optimal martingale measures.